Career skills visualization, tracking and guidance

ABSTRACT

An online career guidance system generates and displays superimposed polygons to a user for a corresponding career. The first polygon may have three to eight vertices, with each vertex representing a career skill. The distance from a focus to each vertex represents a career skill level predetermined to be needed for that career skill for that career. The second polygon has the same number of vertices as the first polygon that represent the same career skills and shares the same focus. The distance from the focus to each vertex of the second polygon represents a user skill level determined from previous achievements of the user. The distance between a career skill level and a user skill level (preferably both normalized to the same scale) visually represents any career skill deficits (or over qualifications) the user may have for the selected career. The online career guidance system may determine, recommend and register the user into one or more classes, clubs and/or projects that are likely to reduce or eliminate any career skill deficits of the user.

CROSS-REFERENCE TO RELATED PATENT APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 15/714,645 titled “Real-World Feedback Network for FeedbackRecipients” filed on Sep. 25, 2017 which is incorporated herein byreference in its entirety.

FIELD OF THE INVENTION

This disclosure relates to an online career guidance system graphicallygenerating and displaying user skill levels superimposed with careerskill levels for one or more selected careers so that the user mayvisually see careers they are qualified for and/or any deficiencies andthe online career guidance system assisting the user in mitigating thedeficiencies.

SUMMARY OF THE INVENTION

The present invention provides systems and methods comprising one ormore server hardware computing devices or client hardware computingdevices, communicatively coupled to a network, and each comprising atleast one processor executing specific computer-executable instructionswithin a memory that, when executed, cause the system to allow aninstructor and/or a learner to shape and request feedback for a projectfrom a plurality of contributors.

The invention is a method for an online educational institute to providefeedback for a project from a plurality of contributors that are part ofa feedback network to a learner and an instructor. The onlineeducational institute may comprise computer code running on one or morehardware servers and may access to one or more hardware databases. Thecomputer code may be, at least, a request modifier and a feedbackaggregator.

The request modifier may electronically transmit a first interface to aclient device of an instructor configured to allow the client device ofthe instructor to enter a default request for feedback for the projectfrom client devices of the plurality of contributors to the learner. Thedefault request may include, as non-limiting examples, a feedback duedate or a project due date, a project title or a project description, asuggested feedback type, a desired volume of feedback and a plurality ofgroups, each group comprising a plurality of potential contributors.

The request modifier may assign a character string to the defaultrequest entered by the instructor. The instructor may communicate,preferably via the client devices of the instructor and the learners,the character string assigned to the default request for feedback fromthe contributors.

The request modifier may electronically receive the character stringfrom the client device of the learner. The request modifier mayelectronically transmit a second interface to the client device of thelearner. In some embodiments, the learner is not given an option tochange the default request for feedback as entered by the instructor.The default request is submitted to the contributors.

In other embodiments, the interface may be configured to allow theclient device of the learner to enter an initial request (possiblealterations or modifications from the default request) for the feedbackfor the project from client devices of the plurality of contributors tothe learner. As non-limiting examples, the initial request may comprisethe feedback due date or the project due date, the project title or theproject description, the suggested feedback type and/or the desiredvolume of feedback and the plurality of groups with each groupcomprising a plurality of potential contributors.

The request modifier may electronically receive a plurality of pastrequests and a corresponding plurality of past feedback from a hardwaredatabase. The request modifier may use the past requests andcorresponding past feedback to determine a plurality of most similarpast requests to the initial request in the plurality of past requests.The request modifier may use the past volume of feedback from the mostsimilar past requests to determine an average volume of feedback for thesimilar past requests.

The request modifier may assign a predicted volume of feedback for thedefault request or initial request based on an average volume of pastfeedback for the plurality of most similar past requests.

The request modifier may modify the default request or the initialrequest to generate a modified request that is preferably predicted toreceive a volume of feedback closer to the desired volume of feedbackthan the predicted volume of feedback for the default request or theinitial request. The request modifier may make adjustments, such aschanging the type of desired feedback or the number of contributorsreceiving the request for feedback, to create a modified request thatwill receive a desired volume of feedback as selected by the instructoror learner.

The request modifier may electronically transmit the modified request tothe plurality of contributors for feedback. One or more of thecontributors receiving the submitted request for feedback may accept therequest and/or provide feedback on the project using an interfacepresented to the contributors.

The feedback aggregator may electronically receive the plurality offeedback from the one or more responding contributors. The feedbackaggregator may generate and electronically transmit an interfacedisplaying the plurality of feedback for the project from the pluralityof contributors to the client device of the learner and a client deviceof an instructor.

In another embodiment, a method may be used to provide career guidanceto the user, i.e., the learner. An online career guidance system mayread from a first database a plurality of user scores. Each user scoremay be determined from a different achievement of the user. Theachievements of the user may be, as non-limiting examples, a previousscore received, possibly as a grade in a class, a score on one rubriccriterion from a final project in a course and/or one or more feedbacksor scores/ratings from one or more online contributors.

The online career guidance system may convert each user score into oneor more user skill levels thereby generating a plurality of user skilllevels. In a preferred embodiment, each user skill level represents apredicted level of a skill of the user for a particular skillpredetermined to be desired in one or more careers. Thus, a score fromone criterion in a rubric in a heavily weighted project in a chemistryclass may be used, preferably with other data points from other classes,to predict a user skill level for the career skills of, as non-limitingexamples, logical reasoning and a user skill level for the career skillof data analysis.

The online career guidance system may normalize the plurality of userskill levels to a desired scale to generate a plurality of normalizeduser skill levels. While any scale may be used, such as between 1-5 or0-100, in a preferred embodiments the plurality of user skill levels arenormalized to the scale of 1-10 to produce normalized user skill levels.Thus, a user score of an “A” in a class may be normalized to a “10,”while a “B” may be normalized to an “8,” and so on. As another example,a user score of a 98% in a class may be normalized to a “10,” while auser score of 72% may be normalized to a “7.” If a user has differentnormalized user skill levels for the same career skill, the differentnormalized user skill levels may be, as non-limiting examples, 1)averaged, 2) only the most recent user score used or 3) only the mostrecent user scores within a most recent time period, such as within thelast year, are averaged to produce a single normalized user skill levelfor the career skill.

It should be appreciated that each user score, user skill level ornormalized user skill level may be weighted differently from other userscores, user skill levels or normalized user skill levels. For example,a final course grade in a business writing course is preferably weightedmore heavily than a score on a rubric criterion of “writing quality”from a marketing course project. Also, the system, over time, may learnwhich inputs are most valid and valuable predictors of success in thatarea.

The online career guidance system may read from a second database aplurality of careers and a plurality of career skill levels for each ofthe plurality of careers. Each career skill level for each of theplurality of careers represents a level of a skill predetermined to bedesired in the career. Thus, if critical thinking is predetermined to bean important or essential career skill for the career of journalism,then critical thinking will be selected as one of the career skills forthat career. In addition, if critical thinking is particularlyimportant, even when compared against other important or essentialcareer skills for that career, it may be assigned a high number on thescale, such as a 10 on a scale of 1-10. If the career skill is importantto the career, and thus one of career skills for that career, but not asimportant as the other selected career skill for that career, a lowernumber, such as a 7 on a scale of 1-10 may be selected as the careerskill level for that given career. In other words, each career may haveits own combination, preferably three to eight, of important careerskills and each career skill may be assigned its own career skill levelfor each career. In some embodiments, the online guidance system maynormalize the plurality of career skill levels, if they are not alreadynormalized, to the same scale as the normalized user skill levels, togenerate a plurality of normalized career skill levels. Normalizing theuser skill levels and career skill levels to the same scale allows theuser skill levels and career skill levels to be graphically displayed tothe user on a client for easy comparison.

As a non-limiting example of selecting one or more careers tographically display to the user on the client, the careers may bepresented in a list or a menu of selectable items. This embodimentprovides the user the most control over which careers are selected andcorresponding graphs are displayed to the user. The user may select andthe online guidance system may receive from the user one or moreselected careers that have the most interest to the user.

In another example of selecting one or more careers to graphicallydisplay to the user on the client device, the online career guidancesystem may compare a plurality of normalized career skill levels foreach career in the plurality of careers with a corresponding pluralityof normalized user skill levels in the plurality of normalized userskill levels. The online career guidance system may select one or morecareers in the plurality of careers with a plurality of normalizedcareer skill levels that most closely matches a corresponding pluralityof normalized user skill levels. This selection method displays to theuser careers that closely align with the user skill level for theimportant career skills of the career.

In another example of selecting one or more careers to graphicallydisplay to the user on the client device, the online career guidancesystem may display a plurality of criteria for selecting one or morecareers in the plurality of careers. Thus, the user may select acriteria of “fully qualified,” “stretch careers,” “random selection,”“closely matches” or “fewest number of career skill deficiencies.” Theonline career guidance system may receive a criteria selected by theuser from the plurality of displayed criteria. The online careerguidance system may compare a plurality of normalized career skilllevels for each career in the plurality of careers with a correspondingplurality of normalized user skill levels in the plurality of normalizeduser skill levels. The online career guidance system may select one ormore careers from the plurality of careers based on 1) the criteriaselected by the user, and 2) the comparing of the plurality ofnormalized career skill levels with the corresponding plurality ofnormalized user skill levels.

The online career guidance system may display on the client to the usera first polygon having a first plurality of vertices and a first focus.Each vertex of the first polygon may be spaced a distance from the firstfocus based on a normalized career skill level in the plurality ofnormalized career skill levels for the selected career. Thus, a highernormalized career skill level will result in a vertex further from thefirst focus while a lower normalized career skill level will result in avertex closer to the first focus. This is a graph illustrating theimportant career skills and the career skill levels predetermined to beneeded for each career skill to perform the selected career. In otherwords, this is a graph that shows the baseline or the goal of where theuser should be striving towards.

The online career guidance system may also display on the client to theuser a second polygon, superimposed with the first polygon, having asecond plurality of vertices and a second focus. The first polygon andthe second polygon preferably have the same number of vertices and thesecond focus is located at the same position on the client as the firstfocus. This superposition or overlaying of the graphs makes it easier tocompare the normalized career skill levels with the normalized userskill levels. Each vertex of the second polygon is spaced a distancefrom the second focus based on a normalized user skill level in theplurality of normalized user skill levels for the selected career. Eachvertex of the second polygon is preferably located on a hypotheticalline defined by the first focus and a corresponding vertex of the firstpolygon. Each vertex of the first polygon or the second polygon may belabeled with a career skill that corresponds with the normalized careerskill level and the normalized user skill level for that vertex.

In some embodiments, the online career guidance system may determine acareer skill deficit between a normalized career skill level in theplurality of normalized career skill levels for the selected career anda corresponding normalized user career skill level in the plurality ofnormalized user career skill levels. The deficit represents a placewhere the user should attempt to improve the normalized user skill levelto more close match or exceed the normalized career skill level so thatthe user is well qualified for the selected displayed career.

The online career guidance system may select an action that haspreviously been determined to reduce the career skill deficit. In apreferred embodiment, the online career guidance system attempts torecommend the fewest possible actions that correct the maximum number ofcareer skill deficits of the user. As an example, the online careerguidance system may recommend taking a series of classes to learn a newlanguage if the user has career skill deficits in communication skillsand creativity and it is known that learning another languageimproves/mitigates/reduces career skill deficits in these career skills.

The online career guidance system may display one or more actions on theclient to the user near the first polygon and the second polygon toreduce the career skill deficit. The actions may be to take one or moreclasses, join one or more clubs, request additional feedback fromcontributors and/or complete one or more projects that are known toimprove a user skill level for the career skill that has the careerskill deficit. In some embodiments, the user skill deficits and theactions performed by the user may be tracked to determine which actionsare the most effective at reducing particular career skill deficits.This information may be used to assist other users having the samecareer skill deficit by recommending the actions to those future usersthat are known to be the most effective at reducing the career skilldeficit.

In some embodiments, the online career guidance system may receive aselection of the displayed action on the client by the user. In otherwords, the user has selected/indicated a desire to perform one of thedisplayed actions so that the user may improve one or more of the careerskill deficits the user has for a selected career. The online careerguidance system may perform a registration process of the user for theaction, such as registering the user into a particular educationalclass, providing information about joining a relevant club orinstructions for performing a related project.

In another embodiment, the online career guidance system may display aplurality of polygons with the same number of vertices sharing a commonfocus with each vertex representing a normalized user skill level. Eachpolygon may represent a different point in time or a time period for thenormalized user skill levels. This allows a user to see their normalizeduser skill levels over time to see if they are making any progress indeveloping their career skills. These polygons may be superimposed,i.e., displayed with or over, a polygon representing normalized careerskill levels that represent career skill levels for a selected career.This allows the user to see how their improvements, over time, if any,are moving them closer to the desired skill levels needed for thevarious career skills of the selected career.

The above features and advantages of the present invention will bebetter understood from the following detailed description taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a system level block diagram for a computerenvironment that allows an instructor or a user on a client device toshape and request feedback over a computer network from a plurality ofcontributors on client devices.

FIG. 2 illustrates a system level block diagram for a computer systemthat may be used in part to allow an instructor and/or a user on aclient device to shape and request feedback over a computer network froma plurality of contributors on client devices.

FIG. 3 illustrates a block diagram of a system that allows an instructorand/or a user on a client device to shape and request feedback over acomputer network from a plurality of contributors on client devices. Arequest modifier may also modify a request for feedback from theinstructor or the learner so as to provide a desired volume of feedback.

FIG. 4 illustrates a flow chart of a method that may be used to allowsan instructor and/or a user on a client device to shape and requestfeedback over a computer network from a plurality of contributors onclient devices. A request modifier may also modify a request forfeedback from the instructor or the learner so as to provide a desiredvolume of feedback.

FIG. 5 illustrates a block diagram of a system that allows an instructorand/or a user on a client device to shape and request feedback over acomputer network from a plurality of contributors on client devices.

FIG. 6 illustrates a flow chart of a method that may be used to allowsan instructor and/or a user on a client device to shape and requestfeedback over a computer network from a plurality of contributors onclient devices.

FIG. 7 illustrates a possible interface that may be displayed by arequest modifier of an online educational institute to a client deviceof an instructor to enter a default request or a client device of alearner to enter an initial request.

FIG. 8 illustrates a possible interface for a learner to enter acharacter string that will auto populate the default request entered bythe instructor into the interface displayed on the client device of thelearner.

FIG. 9 illustrates a possible interface that may be electronicallycommunicated to a client device of a contributor requesting that thecontributor provide feedback to the described project of the identifiedlearner and allowing the contributor to accept or reject the request.

FIG. 10 illustrates a possible interface displayed on a client devicefor a contributor to enter feedback for the project of the learner.

FIG. 11 illustrates a possible interface for a feedback aggregator tosummarize the feedback received from a plurality of contributors to aclient device of an instructor and a learner.

FIG. 12 illustrates a possible interface that may be displayed by theonline educational institute on a client device of a learner to receivefeedback or a client device of a contributor to enter feedback on aproject.

FIG. 13 illustrates a possible interface for the instructor or learnerto receive on a client device an aggregate view and/or summary view ofthe feedback received from a plurality of contributors.

FIG. 14 illustrates a possible interface for the instructor or learnerto receive on a client device that compares the learner's skills in aplurality of areas with the known necessary skills for different careeror employment positions.

FIGS. 15-19 illustrate three different methods for presenting avisualization of normalized user skill levels vs. normalized careerskill levels for a plurality of career skills essential for a particularcareer and recommending one or more actions to mitigate any deficienciesin the normalized user skill levels.

FIG. 20 Illustrates an example of information that may be displayed tothe user, comparing normalized user skill levels with normalized careerskill levels for three different example careers.

DETAILED DESCRIPTION

The present inventions will now be discussed in detail with regard tothe attached drawing figures that were briefly described above. In thefollowing description, numerous specific details are set forthillustrating the Applicant's best mode for practicing the invention andenabling one of ordinary skill in the art to make and use the invention.It will be obvious, however, to one skilled in the art that the presentinvention may be practiced without many of these specific details. Inother instances, well-known machines, structures, and method steps havenot been described in particular detail in order to avoid unnecessarilyobscuring the present invention. Unless otherwise indicated, like partsand method steps are referred to with like reference numerals.

Network

FIG. 1 illustrates a non-limiting example distributed computingenvironment 100, which includes one or more computer server computingdevices 102, one or more client computing devices 106, and othercomponents that may implement certain embodiments and features describedherein. Other devices, such as specialized sensor devices, etc., mayinteract with client 106 and/or server 102. The server 102, client 106,or any other devices may be configured to implement a client-servermodel or any other distributed computing architecture.

Server 102, client 106, and any other disclosed devices may becommunicatively coupled via one or more communication networks 120.Communication network 120 may be any type of network known in the artsupporting data communications. As non-limiting examples, network 120may be a local area network (LAN; e.g., Ethernet, Token-Ring, etc.), awide-area network (e.g., the Internet), an infrared or wireless network,a public switched telephone networks (PSTNs), a virtual network, etc.Network 120 may use any available protocols, such as (e.g., transmissioncontrol protocol/Internet protocol (TCP/IP), systems networkarchitecture (SNA), Internet packet exchange (IPX), Secure Sockets Layer(SSL), Transport Layer Security (TLS), Hypertext Transfer Protocol(HTTP), Secure Hypertext Transfer Protocol (HTTPS), Institute ofElectrical and Electronics (IEEE) 802.11 protocol suite or otherwireless protocols, and the like.

Servers/Clients

The embodiments shown in FIGS. 1-2 are thus one example of a distributedcomputing system and is not intended to be limiting. The subsystems andcomponents within the server 102 and client devices 106 may beimplemented in hardware, firmware, software, or combinations thereof.Various different subsystems and/or components 104 may be implemented onserver 102. Users operating the client devices 106 may initiate one ormore client applications to use services provided by these subsystemsand components. Various different system configurations are possible indifferent distributed computing systems 100 and content distributionnetworks. Server 102 may be configured to run one or more serversoftware applications or services, for example, web-based or cloud-basedservices, to support content distribution and interaction with clientdevices 106. Users operating client devices 106 may in turn utilize oneor more client applications (e.g., virtual client applications) tointeract with server 102 to utilize the services provided by thesecomponents. Client devices 106 may be configured to receive and executeclient applications over one or more networks 120. Such clientapplications may be web browser based applications and/or standalonesoftware applications, such as mobile device applications. Clientdevices 106 may receive client applications from server 102 or fromother application providers (e.g., public or private applicationstores).

Security

As shown in FIG. 1, various security and integration components 108 maybe used to manage communications over network 120 (e.g., a file-basedintegration scheme or a service-based integration scheme). Security andintegration components 108 may implement various security features fordata transmission and storage, such as authenticating users orrestricting access to unknown or unauthorized users,

As non-limiting examples, these security components 108 may comprisededicated hardware, specialized networking components, and/or software(e.g., web servers, authentication servers, firewalls, routers,gateways, load balancers, etc.) within one or more data centers in oneor more physical location and/or operated by one or more entities,and/or may be operated within a cloud infrastructure.

In various implementations, security and integration components 108 maytransmit data between the various devices in the content distributionnetwork 100. Security and integration components 108 also may use securedata transmission protocols and/or encryption (e.g., File TransferProtocol (FTP), Secure File Transfer Protocol (SFTP), and/or Pretty GoodPrivacy (PGP) encryption) for data transfers, etc.).

In some embodiments, the security and integration components 108 mayimplement one or more web services (e.g., cross-domain and/orcross-platform web services) within the content distribution network100, and may be developed for enterprise use in accordance with variousweb service standards (e.g., the Web Service Interoperability (WS-I)guidelines). For example, some web services may provide secureconnections, authentication, and/or confidentiality throughout thenetwork using technologies such as SSL, TLS, HTTP, HTTPS, WS-Securitystandard (providing secure SOAP messages using XML encryption), etc. Inother examples, the security and integration components 108 may includespecialized hardware, network appliances, and the like (e.g.,hardware-accelerated SSL and HTTPS), possibly installed and configuredbetween servers 102 and other network components, for providing secureweb services, thereby allowing any external devices to communicatedirectly with the specialized hardware, network appliances, etc.

Data Stores (Databases)

Computing environment 100 also may include one or more data stores 110,possibly including and/or residing on one or more back-end servers 112,operating in one or more data centers in one or more physical locations,and communicating with one or more other devices within one or morenetworks 120. In some cases, one or more data stores 110 may reside on anon-transitory storage medium within the server 102. In certainembodiments, data stores 110 and back-end servers 112 may reside in astorage-area network (SAN). Access to the data stores may be limited ordenied based on the processes, user credentials, and/or devicesattempting to interact with the data store.

Computer System

With reference now to FIG. 2, a block diagram of an illustrativecomputer system is shown. The system 200 may correspond to any of thecomputing devices, clients, hardware servers or servers of the network100, or any other computing devices described herein. In this example,computer system 200 includes processing units 204 that communicate witha number of peripheral subsystems via a bus subsystem 202. Theseperipheral subsystems include, for example, a storage subsystem 210, anI/O subsystem 226, and a communications subsystem 232.

Processors

One or more processing units 204 may be implemented as one or moreintegrated circuits (e.g., a conventional micro-processor ormicrocontroller), and controls the operation of computer system 200.These processors may include single core and/or multicore (e.g., quadcore, hexa-core, octo-core, ten-core, etc.) processors and processorcaches. These processors 204 may execute a variety of resident softwareprocesses embodied in program code, and may maintain multipleconcurrently executing programs or processes. Processor(s) 204 may alsoinclude one or more specialized processors, (e.g., digital signalprocessors (DSPs), outboard, graphics application-specific, and/or otherprocessors).

Buses

Bus subsystem 202 provides a mechanism for intended communicationbetween the various components and subsystems of computer system 200.Although bus subsystem 202 is shown schematically as a single bus,alternative embodiments of the bus subsystem may utilize multiple buses.Bus subsystem 202 may include a memory bus, memory controller,peripheral bus, and/or local bus using any of a variety of busarchitectures (e.g. Industry Standard Architecture (ISA), Micro ChannelArchitecture (MCA), Enhanced ISA (EISA), Video Electronics StandardsAssociation (VESA), and/or Peripheral Component Interconnect (PCI) bus,possibly implemented as a Mezzanine bus manufactured to the IEEE P1386.1standard).

Input/Output

I/O subsystem 226 may include device controllers 228 for one or moreuser interface input devices and/or user interface output devices,possibly integrated with the computer system 200 (e.g., integratedaudio/video systems, and/or touchscreen displays), or may be separateperipheral devices which are attachable/detachable from the computersystem 200. Input may include keyboard or mouse input, audio input(e.g., spoken commands), motion sensing, gesture recognition (e.g., eyegestures), etc.

Input

As non-limiting examples, input devices may include a keyboard, pointingdevices (e.g., mouse, trackball, and associated input), touchpads, touchscreens, scroll wheels, click wheels, dials, buttons, switches, keypad,audio input devices, voice command recognition systems, microphones,three dimensional (3D) mice, joysticks, pointing sticks, gamepads,graphic tablets, speakers, digital cameras, digital camcorders, portablemedia players, webcams, image scanners, fingerprint scanners, barcodereaders, 3D scanners, 3D printers, laser rangefinders, eye gaze trackingdevices, medical imaging input devices, MIDI keyboards, digital musicalinstruments, and the like.

Output

In general, use of the term “output device” is intended to include allpossible types of devices and mechanisms for outputting information fromcomputer system 200 to a user or other computer. For example, outputdevices may include one or more display subsystems and/or displaydevices that visually convey text, graphics and audio/video information(e.g., cathode ray tube (CRT) displays, flat-panel devices, liquidcrystal display (LCD) or plasma display devices, projection devices,touch screens, etc.), and/or non-visual displays such as audio outputdevices, etc. As non-limiting examples, output devices may include,indicator lights, monitors, printers, speakers, headphones, automotivenavigation systems, plotters, voice output devices, modems, etc.

Memory or Storage Media

Computer system 200 may comprise one or more storage subsystems 210,comprising hardware and software components used for storing data andprogram instructions, such as system memory 218 and computer-readablestorage media 216.

System memory 218 and/or computer-readable storage media 216 may storeprogram instructions that are loadable and executable on processor(s)204. For example, system memory 218 may load and execute an operatingsystem 224, program data 222, server applications, client applications220, Internet browsers, mid-tier applications, etc.

System memory 218 may further store data generated during execution ofthese instructions. System memory 218 may be stored in volatile memory(e.g., random access memory (RAM) 212, including static random accessmemory (SRAM) or dynamic random access memory (DRAM)). RAM 212 maycontain data and/or program modules that are immediately accessible toand/or operated and executed by processing units 204.

System memory 218 may also be stored in non-volatile storage drives 214(e.g., read-only memory (ROM), flash memory, etc.) For example, a basicinput/output system (BIOS), containing the basic routines that help totransfer information between elements within computer system 200 (e.g.,during start-up) may typically be stored in the non-volatile storagedrives 214.

Computer Readable Storage Media

Storage subsystem 210 also may include one or more tangiblecomputer-readable storage media 216 for storing the basic programmingand data constructs that provide the functionality of some embodiments.For example, storage subsystem 210 may include software, programs, codemodules, instructions, etc., that may be executed by a processor 204, inorder to provide the functionality described herein. Data generated fromthe executed software, programs, code, modules, or instructions may bestored within a data storage repository within storage subsystem 210.

Storage subsystem 210 may also include a computer-readable storage mediareader connected to computer-readable storage media 216.Computer-readable storage media 216 may contain program code, orportions of program code. Together and, optionally, in combination withsystem memory 218, computer-readable storage media 216 maycomprehensively represent remote, local, fixed, and/or removable storagedevices plus storage media for temporarily and/or more permanentlycontaining, storing, transmitting, and retrieving computer-readableinformation.

Computer-readable storage media 216 may include any appropriate mediaknown or used in the art, including storage media and communicationmedia, such as but not limited to, volatile and non-volatile, removableand non-removable media implemented in any method or technology forstorage and/or transmission of information. This can include tangiblecomputer-readable storage media such as RAM, ROM, electronicallyerasable programmable ROM (EEPROM), flash memory or other memorytechnology, CD-ROM, digital versatile disk (DVD), or other opticalstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or other tangible computer readablemedia. This can also include nontangible computer-readable media, suchas data signals, data transmissions, or any other medium which can beused to transmit the desired information and which can be accessed bycomputer system 200.

By way of example, computer-readable storage media 216 may include ahard disk drive that reads from or writes to non-removable, nonvolatilemagnetic media, a magnetic disk drive that reads from or writes to aremovable, nonvolatile magnetic disk, and an optical disk drive thatreads from or writes to a removable, nonvolatile optical disk such as aCD ROM, DVD, and Blu-Ray® disk, or other optical media.Computer-readable storage media 216 may include, but is not limited to,Zip® drives, flash memory cards, universal serial bus (USB) flashdrives, secure digital (SD) cards, DVD disks, digital video tape, andthe like. Computer-readable storage media 216 may also include,solid-state drives (SSD) based on non-volatile memory such asflash-memory based SSDs, enterprise flash drives, solid state ROM, andthe like, SSDs based on volatile memory such as solid state RAM, dynamicRAM, static RAM, DRAM-based SSDs, magneto-resistive RAM (MRAM) SSDs, andhybrid SSDs that use a combination of DRAM and flash memory based SSDs.The disk drives and their associated computer-readable media may providenon-volatile storage of computer-readable instructions, data structures,program modules, and other data for computer system 200.

Communication Interface

Communications subsystem 232 may provide a communication interface fromcomputer system 200 and external computing devices via one or morecommunication networks, including local area networks (LANs), wide areanetworks (WANs) (e.g., the Internet), and various wirelesstelecommunications networks. As illustrated in FIG. 2, thecommunications subsystem 232 may include, for example, one or morenetwork interface controllers (NICs) 234, such as Ethernet cards,Asynchronous Transfer Mode NICs, Token Ring NICs, and the like, as wellas one or more wireless communications interfaces 236, such as wirelessnetwork interface controllers (WNICs), wireless network adapters, andthe like. Additionally and/or alternatively, the communicationssubsystem 232 may include one or more modems (telephone, satellite,cable, ISDN), synchronous or asynchronous digital subscriber line (DSL)units, Fire Wire® interfaces, USB® interfaces, and the like.Communications subsystem 236 also may include radio frequency (RF)transceiver components for accessing wireless voice and/or data networks(e.g., using cellular telephone technology, advanced data networktechnology, such as 3G, 4G or EDGE (enhanced data rates for globalevolution), WiFi (IEEE 802.11 family standards, or other mobilecommunication technologies, or any combination thereof), globalpositioning system (GPS) receiver components, and/or other components.

Input Output Streams Etc.

In some embodiments, communications subsystem 232 may also receive inputcommunication in the form of structured and/or unstructured data feeds,event streams, event updates, and the like, on behalf of one or moreusers who may use or access computer system 200. For example,communications subsystem 232 may be configured to receive data feeds inreal-time from users of social networks and/or other communicationservices, web feeds such as Rich Site Summary (RSS) feeds, and/orreal-time updates from one or more third party information sources(e.g., data aggregators). Additionally, communications subsystem 232 maybe configured to receive data in the form of continuous data streams,which may include event streams of real-time events and/or event updates(e.g., sensor data applications, financial tickers, network performancemeasuring tools, clickstream analysis tools, automobile trafficmonitoring, etc.). Communications subsystem 232 may output suchstructured and/or unstructured data feeds, event streams, event updates,and the like to one or more data stores that may be in communicationwith one or more streaming data source computers coupled to computersystem 200.

Connect Components to System

The various physical components of the communications subsystem 232 maybe detachable components coupled to the computer system 200 via acomputer network, a FireWire® bus, or the like, and/or may be physicallyintegrated onto a motherboard of the computer system 200. Communicationssubsystem 232 also may be implemented in whole or in part by software.

Other Variations

Due to the ever-changing nature of computers and networks, thedescription of computer system 200 depicted in the figure is intendedonly as a specific example. Many other configurations having more orfewer components than the system depicted in the figure are possible.For example, customized hardware might also be used and/or particularelements might be implemented in hardware, firmware, software, or acombination. Further, connection to other computing devices, such asnetwork input/output devices, may be employed. Based on the disclosureand teachings provided herein, a person of ordinary skill in the artwill appreciate other ways and/or methods to implement the variousembodiments.

Referring to FIGS. 3-6, systems and methods are illustrated for aninstructor 370, i.e., a data source 370 and a learner 350, i.e., afeedback recipient 350, in an online educational institute 300, i.e., anonline digital feedback network 300, to control the sources, type andquantity of feedback that the instructor 370 and learner 350 receivefrom one or more contributors 360 regarding a project the learner 350 isperforming as a class assignment. It should be appreciated that while,in some embodiments, the instructor 370 and learners 350 may physicallymeet in a brick & mortar building or alternatively only in a virtualonline class, the requests for feedback to the contributors 360 and thefeedback from the contributors 360 to the learners 350 must be receivedvia an online digital electronic computer network (such as theInternet).

The claimed invention is advantageous for the learner 350 as the learner350 is able to manage the sources, type (quality) and quantity offeedback the learner 350 receives from one or more contributors 360. Theimproved feedback enhances the learning experience of the learner andmay also provide the learner (preferably with the consent of thecontributors 360) with a portfolio of projects that have been reviewedby a number of professionals in the same field as the project. Theonline educational institute 300, instructor 370 and learner 350 may alluse the project and the feedback as marketing and promotional tools. Inpreferred embodiments, authorizations and/or notifications have beenobtained from the instructor 370, the learner 350 and/or thecontributors 360 whose material is being used as marketing orpromotional tools.

The claimed invention is also advantageous for the instructor 370 as thefeedback from the contributors 360 enhances the learning experience ofthe learners 350 in the instructor's class, i.e., learning group orgroup. The instructor 370 may request that projects only be turned inwith specified types or volumes of feedback, may request that a draft,feedback on the draft and the final updated project be turned in orframe the requested feedback from the contributors 360. In addition, theaggregate feedback from the contributors 360 may be used by theinstructor 370 in evaluating the course, using the feedback as insightsfrom other experts in the field. Thus, the instructor may update thecourse based on the feedback from the contributors 360.

The claimed invention is also advantageous for the contributors 360 asthe feedback provided by the contributors 360 enhances the contributors'digital reputation and builds networking contacts with the learners 350.The contributors 360 also see the latest academic work being done intheir field and how the latest batch of learners 350 (students) areaddressing the current issues.

An online educational institute 300 may be a legal entity in thebusiness of providing educational classes, such as traditional and/oronline digital classes, that include assigning projects. The onlineeducational institute 300 comprises computer code running on one or morehardware servers 112 and other computer hardware, such as routers, thatpermit the online educational institute 300 to electronicallycommunicate with their associated learners 350, contributors 360 andinstructors 370. Each learner 350, contributor and instructor 370 mayuse a client 108 or client device 108 (such as a smartphone, tablet,laptop or desktop computer) to electronically communicate with theonline educational institute 300.

The computer code of the online educational institute 300 may provideany number of different desired services. In one embodiment, thecomputer code of the online educational institute 300 comprises arequest modifier 320 and a feedback aggregator 330. The request modifier320 and feedback aggregator 330 may comprise one or more computerprograms and/or computer software modules running on the one or morehardware servers 112 of the online educational institute 300.

The request modifier 320 may receive from a learner 350 an initialrequest for feedback from a plurality of contributors 360 and,optionally, a desired volume of feedback from the contributors 360.(Step 400) In other embodiments, a maximum number of contributors 360providing feedback may be included and/or a date after which feedbackcan no longer be received from the contributors 360 may also be includedwith the request to the request modifier 320. The request modifier 320may modify the initial request created by the learner 350 for feedbackfrom a plurality of contributors 360. (Step 410) The request modifier320 may make the modifications for any desired reason, such as therequest from the learner 350 is accompanied by a desired volume (numberof contributors 360 and/or amount of feedback) of feedback from thecontributors 360.

The feedback aggregator 330 may be used to receive, aggregate andtransmit the feedback from the contributors 360 to the learner 350and/or instructor 370. The feedback aggregator 330 may store the requestsubmitted to the contributors 360 and the feedback (or summary of thefeedback, such as the number of contributors 360 that responded and/orthe quantity of the feedback) in a hardware database 110 (also referredto as a data store 110). The request modifier 320 for future requestsfrom the same or different learners 350 may use the request submitted tothe contributors 360 and the feedback stored in the hardware database110 to allow the request modifier 320 to modify requests so that themodified request submitted to the contributors 360 receives a desiredvolume of feedback as indicated by the learner 350. In some embodiments,the request modifier 320 may provide suggestions and/or defaults in auser interface (UI) as part of a user experience (UX) to scaffold therequesting process such that learners 350 are more likely to choosesettings aligned with their feedback needs, i.e., requests withattributes predicted to yield the desired feedback, either by type,volume and/or deadline.

The online educational institute 300 may also comprise a hardwaredatabase 110 that allows the storage and retrieval of past requests (andthe date created or submitted to the contributors 360) and the pastrequests' corresponding feedback (responses). This data may be enteredinto the hardware database 110 by the feedback aggregator 330 afterevery learner request receives feedback. In addition, the requestmodifier 320 may read and use the request and corresponding feedback tomodify requests, either the request itself and/or the request makingexperience, from learners 350 for feedback from contributors 360 on theassumption that similar requests will receive a similar volume offeedback from contributors 360.

In another embodiment, the request modifier 320 of the online educationinstitute 300 may transmit a first interface to a client device 108 ofan instructor 370. The first interface may be configured to allow theclient device 108 of the instructor 370 to enter a default request forthe feedback for a project from client devices 108 of one or morecontributors 360. The default request from the instructor 370 may beintended by the instructor 370 to be used by all of the learners in aclass working on the project. The default request may comprise anyfactors desired by the instructor 370 for the project. As non-limitingexamples, the default request (and all of the other described requestsbelow) may comprise a feedback due date or a project due date, a projecttitle or a project description, a suggested/required feedback type, adesired volume of feedback and/or one or more groups, with each groupcomprising a plurality of potential contributors 360.

The suggested feedback type may comprise any category of feedbackdesired by the instructor 370 and/or learner. As non-limiting examples,the suggested feedback type may be a scoring method of a positive or anegative reaction (thumbs up or thumbs down), a rating on a scale,ratings to various factors listed in a rubric, a drafting comments in asuggestions area and/or an annotated response to the material of theproject. In addition, the suggested feedback type may ask for feedback(using any of the above mentioned scoring methods) on various aspects orfactors of the project. As non-limiting examples, the aspects or factorsof the project may comprise a writing skill, creativity, informationliteracy, visual design and/or critical thinking demonstrated by thelearner 350 in performing the project. In some embodiments, the aspectsor factors of the project may be created by the request modifier 320based on skill maps related to the learner's career interest areas, thatthe learner 350 has received lower scores in the past and/or criteriathe request modifier 320 predicts are relevant based on similar feedbackrequests. The request may request scores and/or comments on each of oneor more aspects or factors of the project.

The request may include a desired volume of feedback. The desired volumeof feedback may indicate a desired number of contributors 360 thatprovide feedback on a project of a learner 350 and/or a quality andquantity of feedback from one or more contributors 360. As anon-limiting example, the desired volume may comprise a request for atleast three contributors to provide feedback comprising rating variousfactors of a project using a supplied rubric. In another embodiment, oneor more types of feedback may have a minimum number of contributorfeedback(s) and/or one or more types of feedback may have a maximumnumber of contributor feedback(s). As an example, the request mayinclude a request for at least two annotated feedbacks and no quickreaction (thumbs up or thumbs down) feedbacks for the project.

The request may also include one or more groups of potentialcontributors 360 that may be contacted and asked to provide feedback tothe learner 350 for the project. The contributors 360 may be organizedinto any desired group or number of groups. As non-limiting examples,the groups available for selection and the selected groups may be fellowlearners, graduate students, other instructors 370, alumni from theonline educational institute 300, alumni from other educationalinstitutes, professional groups (preferably related to the project insome manner), contributor volunteers and/or professionals working in afield related to the project. The request may also include one or morespecific individuals to request for feedback as desired contributors360. In some embodiments, a contributor group request may be required orpreferred. Learners may be given the choice (either always provided oronly after earning a certain status in the system.)

The request may also include a contributor rating, wherein onlycontributors 360 in the selected groups having the contributor rating orhigher receive a request for feedback from the request modifier 320. Inthis manner, the instructor 370 or learner 350 making the request maylimit the feedback to be only from contributors 360 that have previouslybeen rated (possibly by other contributors 360, instructors 370 and/orlearners 350) as having the specified contributor rating. In someembodiments, the request might only include the minimum contributorrating based on a predetermined number of feedback requests of differenttypes.

The contributor rating may be based on any desired metric. Asnon-limiting examples, the contributor rating may be based on metricssuch as: 1) quality of feedback contributions, including learner ratingsand how your feedback relates to the feedback of others with credibilityover time; 2) quantity of feedback contributions over time; 3) types offeedback contributed; 4) feedback types the learners find most usefulare prioritized on an ongoing dynamic basis; 5) a combination ofresponse types would earn status, this may be whatever the system showsstrikes the right balance (for learners and contributors) betweencontributing to the system and with quicker ratings (thumbs up/down) andproviding open-ended response types; 6) demand level of the learner'snetwork, i.e. contributors from more highly requested networks earnhigher status; and 7) willingness to share quote and name of thecontributor 360 to allow the learner 350 to reference that in aportfolio of the learner 350.

The request modifier 320 may assign a character string to the defaultrequest entered by the instructor 370. (Step 600) As an example, theinstructor 370 may enter in the first interface that the feedback duedate is September 1st, the project title is “public relations pitch deckfor new community center,” the suggested feedback type is completing(writing comments in) a suggestions area for the project, the selectedgroups are “graduate students” and “PR professionals” and the desiredvolume of feedback is at least two contributors 360 providing feedback.The request modifier 320 may assign a character string of“PR101Feedback” and display the character string on the client device108 of the instructor 370. The instructor 370 may inform the learners350 in their class, either in person or over their client devices 108,that a default request has been entered into the request modifier 320and the learners 350 can access the default request in the requestmodifier 320 with the character string of “PR101Feedback.” (Step 610)

FIG. 7 illustrates an interface that may be displayed on a client device106 of a learner 350. If the instructor 370 provided a character stringfor a project, the leaner 350 may select, as a non-limiting example, thehyper link 700 so that the learner may enter the character string in afield (such as the pop-up field 800 illustrated in FIG. 8). The learners350, having received the character string for a default request from theinstructor 370, may electronically submit the character string fromtheir client to the request modifier 320. (Step 620) Thus, continuingwith the example, the client device 108 of the learner 350 may transmitthe character string “PR101Feedback” to the request modifier 320.

The request modifier 320 may electronically transmit a second interfaceto the client device 108 of the learner 350. The second interfacepreferably displays to the client device 108 of the learner 350 thedefault request entered by the instructor 370. The default request maybe auto populated into various fields displayed on the second interfaceto the client device 108 of the learner 350. The second interface may beconfigured so that the learner 350 may not change the default request sothat the learner 350 has to use the default request entered by theinstructor 370.

In another embodiment, the second interface may allow the learner 350 tomodify the default request entered by the instructor 370. The secondinterface may allow the client device 108 of the learner 350 to enter aninitial request, different from the default request entered by theinstructor 370, for the feedback for the project from client devices 108of the plurality of contributors 360 to the learner 350. (Step 420) As anon-limiting example, FIG. 7 illustrates an interface that may be usedby a learner 350 to enter an initial request (a similar interface may beused by the instructor 370 to enter the default request). The learner350, using a client device 106, may enter: a project description infield 705; enter or select one or more categories in section or fields710; may enter a course title in field 715; enter or select a projectstatus at menu 720; enter or select a learner confidence level at fieldor menu 725; enter or select a close feedback date in field 730; enteror select one or more career skill focuses of the learner 350 in field740; enter or select one or more preferred or allowed types of feedback,such as quick reaction, rubric-based, descriptive, audio, individual,video, career skills, quick question, annotations on a written projectin section 745; enter a rubric (if one is to be used by contributors 360for the project) in field 750; enter a question to be answered by thecontributors 360 regarding the project in field 755; enter or select oneor more groups from which to select contributors in field 760; enter asearch query to search for a contributor 360 for the project in field765; select zero, one or more recommendations as contributors 360 infield 770; limit the number of selected contributors 360 in filed 775,possibly to those having at least the contributor rating entered orselected in field 780; enter a sharable URL for the contributors 360relevant to the feedback request for the project in field 785; uploadthe project to the request modifier 320 in field 790 to permit access tothe project by the contributors 360; and select submit the initialrequest in field 795 so that the request modifier 320 may submit thecreated and/or modified request to the client devices 106 of thecontributors 360.

In another embodiment, the learner 350 may create an initial requestthat comprises, as non-limiting examples, the feedback due date or theproject due date, the project title or the project description, thesuggested feedback type, the desired volume of feedback and one or moregroups, each group comprising a plurality of potential contributors 360.(Step 400)

In another embodiment the instructor 370 does not create a defaultrequest nor does the instructor 370 provide the learners 350 anassociated character string as each learner 350 from a client device 108may enter and customize an initial request on their own using the secondinterface. In this embodiment, the learner 350 may create an initialrequest that comprises, as non-limiting examples, the feedback due dateor the project due date, the project title or the project description,the suggested feedback type, the desired volume of feedback and one ormore groups, each group comprising a plurality of potential contributors360.

The request modifier 320 may electronically receive a plurality of pastrequests and a corresponding plurality of past feedback from a hardwaredatabase 110. The request modifier 320 may determine a plurality of mostsimilar past requests to the initial request in the plurality of pastrequests. The request modifier 320 may use any desired method ofcomparing and determining which past requests are most similar to theinitial request.

As one possible non-limiting example, the request modifier 320 maydetermine the past requests that are most similar to the initial requestby determining a distance in high dimensional space between the initialrequest and each past request in the plurality of past requests. Forthis example, each factor that may be modified in a request may beassigned a number in a dimension. As non-limiting examples, the due datemay be assigned a number representing the number of days betweensubmitting the request to the contributors 360 and the due date for thefeedback.

The date of the requested feedback may also be assigned a number as theamount of feedback from different contributors 360 may change over time.Thus, the date of each request should also be tracked as a dimension inhigh dimensional space.

The project title or project description may be parsed for subject words(such as marketing) and each subject word may be assigned a number. Inpreferred embodiments, subjects that are closer to each other areassigned numbers that are closer to each other.

The suggested feedback may be assigned the number one for a scoringmethod of a positive or a negative reaction (thumbs up or thumbs down),a two for a rating on a scale, a three for ratings to various factorslisted in a rubric, a four for a drafting comments in a suggestions areaand a five for an annotated response to the material of the project. Insome embodiments, the comments may be in a video and/or an audio format.If audio is used, the audio may be converted to text which could bedisplayed to the learner 350 as comments. Alternatively, or in addition,the video may also be converted to text which could be displayed to thelearner 350.

The one or more selected contributor groups may be assigned a one forfellow learners, a two for graduate students, a three for otherinstructors 370, a four for alumni from the online educational institute300, a five for alumni from other educational institutes, a six forprofessional groups (preferably related to the project in some manner),and a seven for professionals working in a field related to the project.If multiple groups are selected, as a non-limiting example, their valuesmay be added together or a plurality (such as four) of dimensions may bereserved for the groups selected. If fewer groups are selected than thenumber of dimensions reserved for groups, not used dimensions may beassigned a default value, such as zero.

The order of each factor as a dimension may be any desired order, butthe order should be kept consistent between the current requests and allpast requests. In this manner, the current initial request and all pastrequests may be placed in high dimensional space.

The request modifier 320 may select as the plurality of most similarpast requests the past requests in the plurality of past requests (thathave been saved in the hardware database 110) with the smallestdetermined distance in high dimensional space with the initial request.The distance between the initial request and past requests may becalculated by subtracting their positions from each other in highdimensional space.

While any number of similar past requests may be selected, as a specificexample, the 10 closest past requests to the initial request may beselected and their volumes of feedback (number of contributors 360,quality ratings and/or quantity of feedback) may be averaged. Theaverage volume of feedback for the closest past requests to the initialrequest may be taken as a predicted volume of feedback for the initialrequest.

The request modifier 320 may assign a predicted volume of feedback forthe initial request based on an average volume of past feedback for theplurality of most similar past requests. This method assumes thatsimilar requests will receive similar volumes of feedback.

In some embodiments, the request modifier 320 may modify the initialrequest to generate a modified request so that the modified request ispredicted to receive a volume of feedback closer to the desired volumeof feedback than the predicted volume of feedback for the initialrequest.

As an example, if the desired volume of feedback is greater than thepredicted volume of feedback for the initial request, the initialrequest may be modified by any desired means that is likely to increasethe volume of predicted feedback. The initial request may be modified toreceive more feedback by, as non-limiting examples, selecting a largergroup, increasing the number of groups selected and/or selecting groupsthat are known to have more contributors 360 that are likely tocontribute feedback. As another non-limiting example, the initialrequest may be modified so that the requested type of feedback is easierto complete. Thus, an initial request asking for annotated comments maybe modified to ask for a quick reaction, such as a thumbs up or a thumbsdown assuming more contributors 360 are likely to participate if therequired feedback is easier or quicker to produce by the contributors360. In some embodiments, the modified request may be analyzed as theinitial request was analyzed to determine a predicted volume of feedbackfor the modified request. The modified request in this embodiment may berepeatedly modified until the predicted volume of feedback for themodified request is within a predetermined distance from the desiredvolume of feedback or a predetermined time or number of modificationshave been attempted.

As another example, if the desired volume of feedback is less than thepredicted volume of feedback for the initial request, the initialrequest may be modified by any desired means that is likely to decreasethe volume of predicted feedback. The initial request may be modified toreceive less feedback by, as non-limiting examples, selecting a smallergroup, decreasing the number of groups selected and/or selecting groupsthat are known to have fewer contributors 360 that are likely tocontribute feedback. As another non-limiting example, the initialrequest may be modified so that the requested type of feedback is harder(but better for the educational experience of the learner 350) tocomplete. Thus, an initial request asking for a thumbs up or thumbs downreaction may be changed to request annotated comments. In someembodiments, the modified request may be analyzed as the initial requestwas analyzed to determine a predicted volume of feedback for themodified request. The modified request in this embodiment may berepeatedly modified until the predicted volume of feedback for themodified request is within a predetermined distance from the desiredvolume of feedback or a predetermined time or number of modificationshave been attempted. It should be appreciated that the request modifier320 may be automatically making the modifications to the modifiedrequest with or without approval from the learner 350, possibly based onsettings selected or approved by the learner 350.

In preferred embodiments, if the default request or the initial requestfor feedback entered by the learner 350 is modified, the modifiedrequest, preferably highlighting the changes, may be displayed to thelearner 350 for the learner's approval. In preferred embodiments, a highlevel explanation for the reason for the change may be displayed to thelearner 350. If the learner 350 does not approve the modified request,the original default request or the initial request may be used forsubmittal to the selected group(s) of contributors 360. If the learner350 does approve the modified request, the modified request may be usedfor submittal to the selected group(s) of contributors 360. (Step 420)

It should be noted that in some embodiments the instructor's defaultrequest must be used by the learners 350. In other embodiments, theinstructor's default request may be modified by the learners 350. Inother embodiments, the instructor 370 does not enter a default requestand the learner 350 may select the initial request for feedback from thecontributors 360 using menus, fields and/or selection tabs on aninterface displayed on the client of the learner 350. In preferredembodiments, the instructor 370 is able to select whether the defaultrequest entered by the instructor 370 is mandatory, the default requestmay be modified or the learners 350 must enter their own initialrequest.

It should be appreciated that in some embodiments the default requestentered by the instructor 370 may be used without being modified (oreven attempted to be modified) to receive a desired volume of feedbackand/or modified by the learners 350. In other embodiments, an initialrequest entered by a learner 350 may be used without being modified (oreven attempted to be modified) to receive a desired volume of feedbackfrom the contributors 360. Thus, the request modifier 320 mayelectronically transmit the default request, the initial request or themodified request to the selected contributors 360, thereby requestingfeedback from the contributors 360 for the project of the learner 350.(Step 430)

In a preferred embodiment, each selected contributor 106 is given anoption to accept or reject a request for feedback from the learner 350regarding a project. As an example, FIG. 9 illustrates an interfacecommunicated to a client device 106 of a contributor 360 that displaysthe name (and optionally picture or video) of the learner 910, a projecttitle or stage of the project 920, a due date for the feedback on theproject 930, and feedback request instructions regarding the type offeedback the learner 350 is requesting from the contributor 360 as shownin field 940. The interface may also have an accept button 950 and areject button 960 so that the contributor 360 may easily accept orreject the request to provide feedback for the project. In this manner,the request modifier 320 may send requests to additional contributors370 if too many of the contributors 370 are rejecting the request forfeedback for the project for the learner 350.

Each contributor 360 preferably has options, via settings thecontributor 360 has selected, to control the types, frequency, volumeand request categories and characteristics (such as institution, year inschool, major(s), subject areas, etc.) that the contributor 360 iswilling to provide feedback. The system preferably does not requestfeedback from a contributor 360 that is not within the settings orbounds selected by the contributor 360. In other embodiments, eachcontributor 360 may search for outstanding feedback requests andvolunteer to provide feedback for as few or as many feedback requests asdesired by the contributor 360 (as long as the contributor 360 meets theminimum qualifications as requested by the learner 350).

In preferred embodiments, the selected contributors 360 that havereceived a request for feedback may receive an interface from which toreview the project of the learner 350 and from which to enter feedbackregarding the project of the learner 350. The interface preferablyinforms the contributors 360 of the date by which the feedback must beentered and the type of feedback desired by the instructor 370 orlearner 350, e.g., a scoring method of a positive or a negative reaction(thumbs up or thumbs down), a rating on a scale, a ratings for variousfactors listed in a displayed rubric in the interface, comments in asuggestions area or an annotated response to the material of theproject. In some embodiments, only means for entering the desired typeof feedback are presented to the contributors 360 on the interface.

FIG. 10 illustrates an interface for a contributor 360 to enterannotated comments in field 106. The contributor 360 may be identifiedin field 101, a project rating may be identified in field 1020, commentsmay be entered by the contributor 360 in field 1030, the skills asselected by the contributor 360 may be displayed in field 1040 and oneor more predefined action items may be displayed in field 1050 for thecontributor 360 to select, thereby making the review and feedbackprocess for the contributor 360 easier to complete.

If the type of requested feedback is rating various factors listed in arubric, the rubric may be displayed on the interface with means ofrating (such as selecting a number or letter on a scale) each factor(such as creativity or writing skill) listed in the rubric, but in thisembodiment other rating methods would not be displayed on the interface.In this manner a feedback aggregator 330 may electronically receive thedesired type of feedback from the clients of the plurality ofcontributors 360. (Step 440)

FIG. 11 illustrates an interface that may be displayed on a clientdevice 106 of a learner 350 that displays the feedback from one or morecontributors. In this example interface 1100, three contributors 360 arelisted (John Li, Matt Smith and Ella Bakubi), each with differentcontributor ratings and using different types of feedback. The interfacealso provides statistics on the average rating from the contributors 360and the response rate of the contributors 360 for the project of thelearner 350

The feedback aggregator 330 may perform various statistical analysis onthe feedback, such as percentage of contributors 360 receiving thesubmitted request that actually provided feedback and/or the averagescores of the contributors 360 for the project. The feedback aggregator330 may also combine all the comments or suggestions from the differentcontributors 360 onto a single interface or web page, for easy viewingon a client device 106 by the instructor 370 and/or learner 350. Thefeedback aggregator 330 may make available and display the feedback tothe instructor 370 and learner 350 by electronically generating andtransmitting an interface displaying the plurality of feedback from theplurality of contributors 360 to the client device 106 of the learner350 and a client device 106 of an instructor 370 for the project. (Step450) In some embodiments, the learner 350 may tag one or more feedbacksfrom the contributors (or portions thereof) which may be automaticallyshared with the instructor 370.

FIG. 12 illustrates an interface 1200 that lists when the feedback isneeded in field 1210, the selected one or more groups listed in field1220, a description of the project in field 1230, the suggested feedbacktype(s) in field 1240, a project description in field 1250, a feedbackfocus listed in field 1260, a career skills listed in field 1270, and aplurality of feedback from contributors 360 in field 1280. Graphs 1290may also be displayed showing the ratings from contributors 360 forvarious skills of the learner 350 displayed in the performance of theproject. Graphs 1290 may also represent scores for all projectssubmitted by the learner 350 with feedback ratings tied to that skill.In addition or alternatively, a graph may also be used to visualize thedata for each project's related career.

In preferred embodiments, each learner 350 only receives an interfacethat has feedback specifically for the project of the learner 350. Inother words, learners 350 preferably do not see feedback intended forother learners 350. In some embodiments, for a common project performedby a group, i.e., a plurality of learners 350, each member of the groupmay see feedback regarding the common project. The instructor 370 ispreferably able to see the feedback received by all of the learners 350for the project in the instructor's class. In other embodiments, thelearner 350 and/or contributors 370 may have to affirmatively givepermission to the instructor 370 before the instructor may see thecontributors' feedback.

FIG. 13 illustrates a generated interface 1300 displaying feedbacksummary for a plurality of past projects 1330 and 1340. In this exampleillustration electronically communicated to the client device 106 of thelearner 350, a feedback status 1310 is displayed showing, asnon-limiting examples, a percentage of overall feedback rate, apercentage of responses from preferred networks, a number of feedbackrequests, a number of project views and a number of total and starredcontributors 360.

The feedback aggregator 330 may electronically store the submittedrequest (the request actually electronically sent to the contributors360) and the plurality of feedback from the plurality of contributors360 or a summary of the feedback in the hardware database 110. Thus,over time the request modifier 320 and feedback aggregator 330 of theonline educational institute 300 will have access to an ever increasinghardware database 110 of submitted requests and the feedback receivedfrom those submitted requests which may be used to modify future initialrequests to get improved feedback, such as the type of desired feedbackor the volume of desired feedback for the learners 350.

In another embodiment, the feedback from each contributor for eachlearner 350 may be used to calculate various skill levels of the learner350. As non-limiting examples, the skills of writing and/or creativitymay be evaluated and scored by a plurality of contributors 360. Thescores may be normalized based on the past scores of the contributors360 (thus learners 350 with harder grading contributors 360 are notunfairly punished and learners 350 with easier grading contributors 360are not unfairly rewarded) and averaged to give each learner 350 a scorefor a plurality of different skills. The scores and associated skillsmay be displayed to the learner 350.

As a non-limiting example, FIG. 14 illustrates an interface 1400 thatdisplays the scores for various skills of the learner 350 with thevarious skills necessary for a particular career or job title. Aplurality of normalized scores for the learner 350 may be comparedagainst scores necessary, suggested or recommended for differentemployment positions, careers or job titles. Thus, if the learner 350wants a particular career of marketing director, but has a creativewriting score below a listed necessary, suggested or recommendedcreative writing score for the position of a marketing director, thelearner 350 may focus on that particular skill in the future. In otherembodiments, the system may also suggest that a learner include a skill(in the feedback request criteria) that the learner is low in or thatthe system does not have enough data on yet. In addition, the learner350 may discover other employment positions, careers or job titles thatmatch closely or where the learner 350 exceeds the necessary, suggestedor recommended scores and thus may learn of a new career path that maybe well suited to the learner 350 that the learner 350 had notconsidered before and then track skills and pursue feedback on theseattributes over time.

The career skills component of the invention may be used to helplearners 350 track their progress towards their targeted careers. Thecareer skills component may also match learners 350, based on feedbackdata, and usage analytics, to careers that the learners 350 may not haveconsidered or targeted previously. The career skills component mayreference visualizations of skills needed in a given career and trackand promote learner 350 progress towards those skills. Learner 350skills relative to desired level on the career skills graph will beearned through feedback requests that include rating specific skills thelearner needs and responses that address specific skills (for examplevia NLP). Just as in gaming, a learner 350 may start with zero points onthe career skills graph and earn progress on it based on feedbackrequests and metrics. Skills (perhaps the top 2 or 3) for which moreevidence is needed will be suggested as a default when the learner 350submits a new request; there may be an option to modify the new requestbefore sending the request out. Progress along a certain dimension ofthe career skills graph may be calculated based on metrics such asratings and responses specific to a given skill (e.g., critical thinkingas a criterion within a rubric) and the credibility and status of thoseproviding the feedback.

FIGS. 15-19 illustrate other methods for providing visual and textualcareer guidance to a user. Specifically, the methods display, for aselected career, a graph representing normalized career skill levels2020 superimposed, i.e., displayed at the same time and in the samelocation, with a graph representing normalized user skill levels 2030for a plurality of career skills 2010 that are important for theselected career. In addition, the methods allow for graphs to bedisplayed for a plurality of different selected careers. The methods mayalso display one or more actions 2080, 2081, 2082 to the user based onany deficiencies 2060 between the normalized user skill levels 2030 andthe normalized career skill levels 2020 for one or more careers toassist the user in improving the user's skills so that the user maybecome qualified for the one or more selected careers.

Referring specifically to FIG. 20, three sets of graphs for threedifferent selected careers are illustrated. The three different selectedcareers illustrated as non-limiting examples in FIG. 20 are PR (PublicRelations) 2000, Journalism 2001 and Marketing 2002. While the threedifferent selected careers are general, the selected careers may be moreor less specific as desired compared to these non-limiting examples.

Each set of graphs may be, as non-limiting examples, a spidergram,spider diagram or a radar chart. The set of graphs may be a graphicaldisplay of multivariate data in the form of a two-dimensional (or usingvirtual reality technology may create three-dimensional) chart of threeor more quantitative variables represented on axes or vertices on apolygon starting from the same point or focus. The position and/or angleof the axes might not convey data, while the data is conveyed by thelength of each axes or vertex from the point or focus.

It should be appreciated that any number of sets of graphs for selectedcareers may be generated and used, but are preferably at least one, andno more than eight, displayed selected careers. Generating anddisplaying fewer selected careers allows the graphs to be larger in sizeand thus easier to view, while generating and displaying more selectedcareers provides information regarding additional selected careers tothe user. A database may be used to store a plurality of differentcareers that may be selected from in order to select the one or moreselected careers.

Each selected career has a plurality of different career skills 2010that have been previously determined to be important or essential inorder to be well qualified for that selected career. It should beappreciated that the career skills 2010 that may be essential in oneselected career may not be needed at all in a different selected career.Thus, each selected career has a combination of career skills 2010 thatare the most important career skills 2010 for that selected career.While each career may have any number of career skills 2010 that areessential for the career, in preferred embodiments, each career isassigned three to eight different career skills 2010 that are the mostimportant career skills 2010 for being effective in that particularcareer.

As non-limiting examples, FIG. 20 illustrates the career PR 2000 asrequiring or recommending the career skills 2010 of Writing, CriticalThinking, Visual Design, Information Literacy and Creativity; the careerJournalism 2001 having the career skills 2010 of Web Writing, CopyEditing, Critical Thinking, Data Management and Photography; and thecareer Marketing 2002 having the career skills 2010 of Copy Writing,Statistics, Data Management, Market Research and Creative Advertising.Thus, according to the example illustrated in FIG. 20, the career of PR2000 has as its most important skills the career skills 2010 of Writing,Critical Thinking, Visual Design, Information Literacy and Creativity.Also according to FIG. 20, the career of Journalism 2001 has as its mostimportant skills the career skills 2010 of Web Writing, Copy Editing,Critical Thinking, Data Management and Photography. Further, accordingto FIG. 20, the career of Marketing 2002 has as its most importantskills the career skills 2010 of Copy Writing, Statistics, DataManagement, Market Research and Creative Advertising.

Each career skill 2010 for a particular career may also be assigned acareer skill level. The career skill level 2020 indicates how talented,skilled or capable a user should be at that career skill 2010 to beeffective working at or being employed in that career. This reflects thefact that career skills 2010 needed for a particular career may not allbe needed at the same level. In other words, some career skills 2010,even compared to other important career skills 2010 for the same career,may be needed at a higher or a lower level than other career skills2010. In addition, the same career skill 2010, for example writing, maybe important for multiple careers (but not all careers), but may berequired at different career skill levels 2020 for each of the multiplecareers.

As a non-limiting example, FIG. 20 illustrates that the career PR 2000has a recommended career skill level 2020 of 10 for the career skill2010 of Writing, the career skill level 2020 of 10 for the career skill2010 of Critical Thinking, the career skill level 2020 of 9 for thecareer skill 2010 of Visual Design, the career skill level 2020 of 10for the career skill 2010 of Information Literacy and the career skilllevel 2020 of 6 for the career skill 2010 of Creativity.

As another non-limiting example, FIG. 20 also illustrates that thecareer Journalism 2001 has a recommended career skill level 2020 of 8for the career skill 2010 of Web Writing, a career skill level 2020 of10 for the career skill 2010 of Copy Editing, a career skill level 2020of 9 for the career skill 2010 of Critical Thinking, a career skilllevel 2020 of 9 for the career skill 2010 of Data Management and acareer skill level 2020 of 8 for the career skill 2010 of Photography.

As yet another non-limiting example, FIG. 20 also illustrates that thecareer Marketing 2002 has a recommended career skill level 2020 of 10for the career skill 2010 of Copy Writing, a career skill level 2020 of9 for the career skill 2010 of Statistics, a career skill level 2020 of9 for the career skill 2010 of Data Management, a career skill level2020 of 8 for the career skill 2010 of Market Research and a careerskill level 2020 of 8 for the career skill 2010 of Creative Advertising.

Each selected career may have a set of graphs, such as one graph for thenormalized career skill levels 2020 and one graph for the normalizeduser skill levels 2030 for the same plurality of career skills 2010.Each graph in the set of graphs for the selected career is preferablyconstructed so that each graph is a polygon with each vertex of thepolygon corresponding to a different one of the plurality of careerskills 2010. Each graph in the set of graphs has the same number ofvertices so that each vertex represents one career skill 2010. Eachgraph in the set of graphs for the career also preferably share a commonposition for a focus 2040.

In a preferred embodiment, the focus 2040 and each vertex for the samecareer skill 2010 for each graph in the set of graphs are on a singleline. This results in one spoke, which may or may not be displayed onthe graphs, for each career skill 2010 radiating out from the focus 2040to the different vertices of the polygons. The spokes/verticesrepresenting the career skills 2010 are preferably displayed radiallysymmetrical, i.e., equally spaced apart from each other, with the otherspokes/vertices by the same angle from the focus 2040.

The distance from a focus 2040 of a graph, representing normalizedcareer skill levels 2020, to a vertex of the graph corresponds to anormalized career skill level. The distance from a focus 2040 of agraph, representing normalized user skill levels, to a vertex of thegraph corresponds to a normalized user skill level. In other words,higher normalized career skill levels 2020 and higher normalized userskill levels 2030 are represented on the graphs by longer distances fromthe focus 2040 to a corresponding vertex, while lower normalized careerskill levels 2020 and lower normalized user skill levels 2030 arerepresented by shorter distances from the focus 2040 to a correspondingvertex. This allows the user to easily compare their skill levelsagainst corresponding career skill levels 2020 necessary for theselected career. This also allows an online guidance system to recommendone or more actions 2080, 2081, 2082 to the user based on anydeficiencies 2060 (career skills 2010 where the normalized career skilllevel 2020 is greater than the normalized user skill level) to any ofthe displayed career skills 2010.

As an example, a user deficiency 2050 is illustrated for the careerskill 2010 Copy Writing for the career Marketing 2002 as the user has auser skill level 2030 of 7 while the recommended career skill level 2020is 10. This user deficiency 2050 indicates the user may need to takesome action 2080, 2081, 2082 (such as taking a class or performing aproject) to mitigate/reduce/eliminate the user deficiency 2050. Asanother example, a user over-qualification 2060 is illustrated for thecareer skill 2010 Creativity for the career PR 2000 as the user has auser skill level 2030 of 9 while the recommended career skill level 2020is 6. This user over-qualification 2060 indicates that the user nolonger needs to take any action 2080, 2081, 2082 regarding this careerskill 2010 and instead should focus on improving other career skills2010 that have a user deficiency 2050.

The career skill levels 2020 are preferably normalized to a desiredscale to produce a plurality of normalized career skill levels 2020 foreach career that are displayed. While any scale may be used, in theillustrated examples in FIG. 20, the career skill levels 2020 arenormalized to be on a scale of one to 10. On this scale, a one for acareer skill level 2020 may be used to indicate that the career skill2010 is barely needed for the selected career while a career skill level2020 of 10 may be used to indicate that the career skill 2010 is neededat a very high level for the user to be effective in the selectedcareer.

The career skill levels 2020 are preferably normalized before beingstored in a database so that the normalization process only has to becalculated/performed one time. However, in another embodiment, thecareer skill levels 2020 are stored in the database not normalized andthe career skill levels 2020 are normalized every time after being readfrom the database, but before being used to produce the illustratedgraphs.

It should be appreciated that the selected careers, career skills 2010,normalized career skill levels 2020 and normalized user skill levels2030 displayed in FIG. 20 are only displayed as examples for the purposeof explaining the invention and the invention is not limited to thespecific displayed careers, career skills 2010, normalized career skilllevels 2020 or normalized user skill levels.

In an embodiment of the invention, an online career guidance system mayauthenticate and log user in to a user account. (Step 1500) Any type ofauthentication method may be used, such as knowledge of a user accountand corresponding password and/or biometric information of the user.

The online career guidance system may read from a first database aplurality of user scores, wherein each user score is determined from adifferent achievement of the user. (Step 1510) As a non-limitingexample, a user score may be a grade (such as a letter grade or anumerical grade) received from a particular class or any otherachievement of the user that may be used to calculate a user skill level2030 for a particular career skill 2010. The online career guidancesystem may retrieve data from a table or database to determine that theuser score may be used to calculate one or more user skill levels 2030for one or more corresponding career skills 2010. Thus, a user score,such as a grade in one class, may be used to calculate one or moredifferent career skill levels 2020 for a corresponding one or moredifferent career skills 2010. (Step 1520)

As an example, the user may have received a user score, e.g., a grade,in a writing class. The class is preferably one of many classes that haspreviously been reviewed to determine that the user scores in the classmay be used to determine a user skill level 2030 in one or more careerskills 2010. As a non-limiting example, the user score in a writingclass may be used to determine a user skill level 2030 in the careerskill 2010 of Writing and a user skill level 2030 in the career skill2010 of Researching.

Letter grades may be normalized by any desired method. As an example, an“A” may be normalized to a “10,” a “B” may be normalized to an “8,” a“C” may be normalized to a “6,” a “D” may be normalized to a “4” and an“F” may be normalized to a “2.”

As another set of examples, a user score of “97” in a class may benormalized to a “10,” a user score of “91” may be normalized to a “9,” auser score of “77” may be normalized to an “8.” This normalizationmethod may be performed by dividing the user score by 10 and thenrounding the result, i.e., quotient, to the nearest integer to generatea user skill level.

In other embodiments, each user score, user skill level and/ornormalized user skill level is weighted based on importance, validity,degree to which it is related to the career skill, source, recency, typeof assessment or action and/or a course from which the score was takenin context of a larger program, e.g., data analysis input from CollegeMath 101 is weaker (weighted less) than data analysis input from Mathfor Data Science 2.

This process may be continued for each user score for the user that isavailable to the online career guidance system. In the event the userhas two or more different normalized user skill levels 2030 for the samecareer skill 2010, the different normalized user skill levels 2030 maybe: 1) averaged; 2) only the normalized user skill level 2030 based onthe most recently received user score is used; or 3) only the normalizeduser skill levels 2030 based on received user score(s) within a mostrecent time period, such as within the last year, are averaged together.This results in each career skill 2010 for the user having only onenormalized user skill level.

In this manner the online career guidance system may generate aplurality of normalized user skill levels 2030 for a correspondingplurality of career skills 2010 for the user. (Step 1530) Eachnormalized user skill level 2030 represents a predicted level of a skillof the user for a particular career skill 2010 predetermined to bedesired in one or more careers.

The online career guidance system may read from a second database (whichcould be the same database as the first database) a plurality of careersand a plurality of career skill levels 2020 for each of the plurality ofcareers. (Step 1540) Each career skill level 2020 for each of theplurality of careers represents a level of a skill predetermined to bedesired, recommended and/or essential in effectively performing thecareer. The career skill levels 2020 are preferably normalized to matchthe scale of the user skill levels 2030 to produce normalized careerskill levels 2020. In a preferred embodiment, the user skill levels 2030and career skill levels 2020 are both normalized to be on the scale of 1to 10 or 0 to 10. (Step 1550)

The online career guidance system may have previously normalized thecareer skill levels 2020 prior to storage on the second database so thatthe career skill levels 2020 are already normalized upon reading themfrom the second database or the online career guidance system maynormalize the plurality of career skill levels 2020 every time afterreading them from the second database to produce the plurality ofnormalized career skill levels 2020.

The online career guidance system may select which career(s) to displayon a client to the user using any desired method. As a non-limitingmethod of selecting a career(s), the online career guidance system maydisplay a plurality of careers to the user on the client device. (Step1600) The plurality of careers may be in a displayed list or may bedisplayed in a drop down menu on the client so that the user is able toselect one or more of the plurality of careers. The online careerguidance system may receive from the user the selection of a career (ora plurality of selected careers) in the plurality of careers displayedto the user. This method allows the user to directly select thecareer(s) that will be used to compare the normalized user skill levels2030 against the normalized career skill levels 2020 for the selectedcareer(s). (Step 1610) In another embodiment, a search/discovery optionmay be displayed for the user to enter a career name like CorporateTrainer. The user may then be displayed one or more careers related tothe entered career, possibly such as Museum Education.

As another non-limiting method of selecting a career(s), the onlinecareer guidance system may compare the plurality of normalized careerskill levels 2020 for each career in the plurality of careers with acorresponding plurality of normalized user skill levels 2030 in theplurality of normalized user skill levels. (Step 1700) The comparisonmay be performed by subtracting the normalized user skill levels 2030from the corresponding normalized career skill levels 2020 orsubtracting the normalized career skill levels 2020 from thecorresponding normalized user skill levels. The online career guidancesystem may select a career (or a plurality of selected careers) in theplurality of careers with a plurality of normalized career skill levels2020 that most closely matches (has the smallest absolute difference) acorresponding plurality of normalized user skill levels. This methodallows the online career guidance system to select the career(s) thatare the most closely aligned to the user's current qualifications, i.e.,normalized user skill levels. In another embodiment, the online careerguidance system may select the career(s) that produces the greatestdifference when subtracting the normalized user skill levels 2030 fromthe corresponding normalized career skill levels 2020. This methodallows the online career guidance system to select the career(s) forwhich the user is the most qualified (or perhaps even the mostoverqualified). (Step 1710)

As another non-limiting method of selecting a career(s), the onlinecareer guidance system may display on the client a plurality of criteriafor selecting a career in the plurality of careers. (Step 1800) Asnon-limiting examples, the displayed criteria for the user to select maybe “only careers fully qualified for,” “careers most closely qualifiedfor,” “careers with the fewest career skills with deficits,” “careersmost overqualified for,” or “stretch careers.”

The user may select one of the criteria to use in selecting a career(s).(Step 1810) The online career guidance system may receive the criteriaselected by the user from the plurality of criteria. The online careerguidance system may compare a plurality of normalized career skilllevels 2020 for each career in the plurality of careers with acorresponding plurality of normalized user skill levels 2030 in theplurality of normalized user skill levels. (Step 1820) The online careerguidance system may select a career(s) from the plurality of careersbased on the criteria selected by the user and the comparing of theplurality of normalized career skill levels 2020 with the correspondingplurality of normalized user skill levels. (Step 1830)

Thus, if the user selects the criteria of “most overqualified for,” theonline career guidance system may compare the normalized user skilllevels 2030 against various career skill levels 2020 for differentcareers to determine the career(s) that the user is the most (orpossibly the top two, three or four) over qualified for. This may bedetermined, as an example, by subtracting the user skill levels 2030from the corresponding career skill levels 2020, adding the differencesand then dividing the total by the number of career skills 2010 used forthat career (to account for the fact that careers may have differentnumbers of career skills 2010 that are deemed important or essential forthe career). This produces an average deficiency (or over qualification2050) per career skill 2010 for each career. The career(s) with thelowest deficiency and/or highest over qualification 2050 may be selectedfor display to the user based on the criteria “most overqualified for”being selected by the user.

The online career guidance system may then display for each selectedcareer a set of graphs, preferably sharing a common focus 2040 orreference point, comprising a graph representing the normalized careerskill levels 2020 simultaneously (Step 1900) superimposed on or with agraph representing the normalized user skill levels 2030 (Step 1910).The representation of the two graphs visually displays to the user whichof the selected careers the user is well qualified for and which of theselected careers the user would need to improve on one or more careerskills 2010.

Each graph in a set of graphs for a selected career may be a polygonwith three to eight vertices that shares a common focus 2040 orreference point with the other graphs. As an example, the online careerguidance system may display on the client to the user a first polygonhaving a first plurality of vertices and a first focus 2040. Each vertexof the first polygon is preferably spaced a distance from the firstfocus 2040 based on a normalized career skill level 2020 in theplurality of normalized career skill levels 2020 for the selectedcareer. The greater the normalized career skill level, the farther thedistance the corresponding vertex is from the focus 2040.

The online career guidance system may also simultaneously display on theclient to the user a second polygon, superimposed with the firstpolygon, having a second plurality of vertices and a second focus 2040.The first polygon and the second polygon have the same number ofvertices and the second focus 2040 is located at the same position onthe client as the first focus 2040.

Each vertex of the second polygon is spaced a distance from the secondfocus 2040 based on a normalized user skill level 2030 in the pluralityof normalized user skill levels 2030 for the selected career. Thegreater the normalized career skill level, the farther the distance thecorresponding vertex is from the focus 2040.

Each vertex of the second polygon may be located on a hypothetical linedefined by the first focus 2040 and a corresponding vertex of the firstpolygon. Thus, some vertices may be closer and some vertices may befarther away from the focus 2040 or reference point, so that the firstpolygon and/or the second polygon are irregular polygons.

Each vertex of the first polygon or the second polygon may be labeledwith a career skill 2010 that corresponds with the normalized careerskill level 2020 and the normalized user skill level 2030 for thatvertex. As a specific example, the vertices for the polygons for thecareer PR 2000 in FIG. 20 are labeled “Writing,” “Critical Thinking,”“Visual Design,” “Information Literacy,” and “Creativity.”

In some embodiments, normalized user skill levels 2030 may be determinedat various points in time. This allows a plurality of graphs to bedisplayed that show the user's progress (hopefully) in various careerskills 2010 over time. Thus, the online career guidance system maydetermine a plurality of normalized prior user skill levels, whereineach normalized prior skill level is a skill level of the user at a sameprior date. The online career guidance system may display on the clientto the user a third polygon, superimposed with the first polygon and thesecond polygon, having a third plurality of vertices and a third focus2040. The first polygon, the second polygon and the third polygon mayhave the same number of vertices and the third focus 2040 is located inthe same position on the client as the first focus 2040 and the secondfocus 2040. Each vertex of the third polygon is spaced a distance fromthe third focus 2040 based on a normalized prior user skill level 2030in the plurality of normalized prior user skill levels 2030 for theselected career. Each vertex of the third polygon may be located on ahypothetical line defined by the first focus 2040 and a correspondingvertex of the first polygon. This method may be used to illustrate thegrowth of the user's skills in the various career skills 2010 as thethird polygon will typically be the smallest, with the second polygon,representing the more recent skill levels, being larger, with thedifference thereby representing any improvements of the user over timefor the various career skills 2010.

The online career guidance system may determine a career skill deficitbetween a normalized career skill level 2020 in the plurality ofnormalized career skill levels 2020 for the selected career and acorresponding normalized user career skill level 2020 in the pluralityof normalized user career skill levels 2020. (Step 1920) As an example,a user deficit 2050 is illustrated for the career skill 2010 of CopyWriting for the career Marketing 2002. In this example, the career skill2010 Copy Writing has a normalized career skill level 2020 of 10 whilethe user only has a normalized user skill level 2030 of 7. Thus, theuser has a deficit of 10 minus 7 or 3 for the career skill 2010 of CopyWriting.

The online career guidance system may have a database that stores one ormore actions 2080, 2081, 2082 that are known to reduce, eliminate ormitigate deficits in one or more of the career skills 2010. Thus, theonline career guidance system may have one or more classes, clubs,organizations or projects that the online career guidance system mayrecommend to the user based on the various career skill deficit(s) foreach selected career. As another example, the online career guidancesystem may recommend that the user pursue outside feedback from five ormore people on three or more artifacts (work products) that would thenprovide inputs for the spider graph that are relevant to the skill inquestion.

The online career guidance system may select one or more actions 2080,2081, 2082, i.e., classes and/or projects, to reduce any deficiencies2060. (Step 1930) In a preferred embodiment, the online career guidancesystem selects the fewest possible class(es) and/or project(s) that,based on past users, will reduce, eliminate or mitigate all of thedeficits. As an example, if the user has a deficit in a career skill #1and a career skill #2 and a particular class is known to improve acorresponding normalized user skill level 2030 for the career skill #1and the career skill #2, the online career guidance system may recommendthe one class in an attempt to remedy both deficiencies 2060.

In some cases, where a user deficiency is substantial, the online careerguidance system may, based on previous experience, determine that asingle class and/or project will not fully correct the user deficiencyand thus may recommend two or more classes/projects to mitigate the onesubstantial deficiency. The online career guidance system may displaythe action 2080, 2081, 2082, comprising one or more classes and/orprojects, on the client to the user near the first polygon and thesecond polygon. (Step 1940) As non-limiting examples, the action 2080 of“Take Class PR 201” is illustrated for the career PR, the action 2081 of“Take Class Journalism 350 and Project—Create a Website illustratingPhotographs that are Stored in a Database” is illustrated for the careerJournalism 2001 and the action 2082 of “Take Classes Marketing 403,Marketing 404 and English 350” is illustrated for the career Marketing2002.

In some embodiments, the online career guidance system in recommending amitigation action may base any actions on a profile as the user. Asnon-limiting examples, the user profile may be used to determine thatthe user has a preference for certain actions over other actions, suchas more willing to request additional feedback verses taking anothercourse or performing an action that is free verses performing an actionthat requires additional costs for the user.

In some embodiments, the displayed actions 2080, 2081, 2082 may beselectable by the user. In these cases the online career guidance systemmay receive a selection of the action 2080, 2081, 2082 on the clientfrom the user. The online career guidance system may perform aregistration process of the user for the action 2080, 2081, 2082. As anexample, if the user selects the displayed recommended action 2080 of“Take Class PR 201,” the online career guidance system may start theprocess and actually register the user in to the class PR 201.

The actions 2080, 2081, 2082 recommended and their effectiveness, afterbeing performed, in changing one or more normalized user skill levels2030 for various career skills 2010 may be determined and stored in adatabase. Thus, later users may benefit as the online career guidancesystem may recommend the actions 2080, 2081, 2082 that have been shownto be the most effective at improving a career skill 2010 to the userthat needs to improve on the career skills 2010. This allows the onlinecareer guidance system to become better at suggesting actions 2080,2081, 2082 over time.

In some embodiments, the user may view a 3-dimensional model of the setof graphs for each selected career. Computer software may be used tocalculate a mathematical representation of the surface of the set ofgraphs for each selected career. The 3-dimensional model may bedisplayed as a 2-dimensional image through a process of 3D rendering orused in a computer simulation to view the 3-dimensional model of the setof graphs as a physical object. The 3-dimensional model of the set ofgraphs may be viewed using any desired technique of generating virtualreality. As a non-limiting example, the user may use special glasses orgoggles to view the 3-dimensional image.

In another non-limiting example, the set of graphs for each selectedcareer may be displayed as a hologram created by the interference oflight beams from a laser or other coherent light source. The3-dimensional model of the set of graphs for each selected career mayfurther assist or enhance the user's experience in determining whichcareer skills 2010 for different careers the user is qualified for andwhich career skills 2010 needs further development through additionaltraining, which may be suggested to the user.

In another embodiment, an application, related widget or extension mayallow a user to view and compare sets of graphs for posted jobs withtheir own sets of graphs. The user may also search for jobs that mostclosely match their own sets of graphs.

In another embodiment, there could be a mechanism for users to see theskills graphs of their friends or contacts on Linkedin™ or the aggregateof skills of people in certain types of roles, e.g., a set of graphsrepresenting the average for all of the robotics engineers at Google™.

Other embodiments and uses of the above inventions will be apparent tothose having ordinary skill in the art upon consideration of thespecification and practice of the invention disclosed herein. Thespecification and examples given should be considered exemplary only,and it is contemplated that the appended claims will cover any othersuch embodiments or modifications as fall within the true scope of theinvention.

The Abstract accompanying this specification is provided to enable theUnited States Patent and Trademark Office and the public generally todetermine quickly from a cursory inspection the nature and gist of thetechnical disclosure and in no way intended for defining, determining,or limiting the present invention or any of its embodiments.

The invention claimed is:
 1. An online career guidance system forproviding career guidance to a user, comprising: a plurality of servers;a first database; a second database; and a communication network inelectronic communication between the plurality of servers and the firstdatabase and the second database; wherein the online career guidancesystem is configured to: authenticate and login a user in to a useraccount; read from the first database a plurality of user scores,wherein each user score is determined from a different achievement ofthe user; convert each user score in the plurality of user scores intoone or more user skill levels thereby generating a plurality of userskill levels, wherein each user skill level represents a predicted levelof a skill of the user for a particular skill predetermined to bedesired in one or more careers; normalize the plurality of user skilllevels to a desired scale to generate a plurality of normalized userskill levels; read from the second database a plurality of careers and aplurality of career skill levels for each of the plurality of careers,wherein each career skill level for each of the plurality of careersrepresents a level of a skill predetermined to be desired in the career;normalize the plurality of career skill levels to generate a pluralityof normalized career skill levels; display the plurality of careers tothe user on the client; receive from the user a selection of a career inthe plurality of careers displayed to the user; display on the client tothe user a first polygon having a first plurality of vertices and afirst focus, wherein each vertex of the first polygon is spaced adistance from the first focus based on a normalized career skill levelin the plurality of normalized career skill levels for the selectedcareer; display on the client to the user a second polygon, superimposedwith the first polygon, having a second plurality of vertices and asecond focus, wherein the first polygon and the second polygon have thesame number of vertices and the second focus is located at the sameposition on the client as the first focus, wherein each vertex of thesecond polygon is spaced a distance from the second focus based on anormalized user skill level in the plurality of normalized user skilllevels for the selected career, wherein each vertex of the secondpolygon is located on a hypothetical line defined by the first focus anda corresponding vertex of the first polygon and wherein each vertex ofthe first polygon or the second polygon is labeled with a career skillthat corresponds with the normalized career skill level and thenormalized user skill level for that vertex; determine a career skilldeficit between a normalized career skill level in the plurality ofnormalized career skill levels for the selected career and acorresponding normalized user career skill level in the plurality ofnormalized user career skill levels; select an action that haspreviously been determined to reduce the career skill deficit; anddetermine and display an action on the client to the user near the firstpolygon and the second polygon.
 2. The system of claim 1, wherein theonline career guidance system is further configured to: receive aselection of the action on the client by the user; and perform aregistration process of the user for the action.
 3. The system of claim2, wherein the action is an online educational class.
 4. The system ofclaim 1, wherein the first polygon and the second polygon are irregularpolygons.
 5. The system of claim 1, wherein each career in the pluralityof careers is associated with three to eight different skillspredetermined to be desired in the career.
 6. The system of claim 1,wherein the online career guidance system is further configured to:determine a plurality of normalized prior user skill levels, whereineach normalized prior skill level is a skill level of the user at a sameprior date; and generate and display on the client to the user a thirdpolygon, superimposed with the first polygon and the second polygon,having a third plurality of vertices and a third focus, wherein thefirst polygon, the second polygon and the third polygon have the samenumber of vertices and the third focus is located in the same positionon the client as the first focus and the second focus, wherein eachvertex of the third polygon is spaced a distance from the third focusbased on a normalized prior user skill level in the plurality ofnormalized prior user skill levels for the selected career, wherein eachvertex of the third polygon is located on a hypothetical line defined bythe first focus and a corresponding vertex of the first polygon.
 7. Thesystem of claim 1, wherein the online guidance system is furtherconfigured to: generate and display a first plurality of polygonscomprising vertices that represent a normalized career skill level,wherein each of the first plurality of polygons represents a differentcareer and is superimposed with one of a corresponding second pluralityof polygons comprising vertices that represent a normalized user skilllevel, wherein each polygon, as compared to the other polygons, in thefirst plurality of polygons comprises a plurality of verticesrepresenting a different combination of career skills predetermined tobe desired in the career.
 8. A method of providing career guidance to auser, comprising the steps of: authenticating and logging by an onlineguidance system a user in to a user account; reading from a firstdatabase by the online career guidance system a plurality of userscores, wherein each user score is determined from a differentachievement of the user; converting each user score in the plurality ofuser scores into one or more user skill levels thereby generating aplurality of user skill levels, wherein each user skill level representsa predicted level of a skill of the user for a particular skillpredetermined to be desired in one or more careers; normalizing by theonline career guidance system the plurality of user skill levels to adesired scale to generate a plurality of normalized user skill levels;reading from a second database by the online career guidance system aplurality of careers and a plurality of career skill levels for each ofthe plurality of careers, wherein each career skill level for each ofthe plurality of careers represents a level of a skill predetermined tobe desired in the career; normalizing by the online career guidancesystem the plurality of career skill levels to generate a plurality ofnormalized career skill levels; comparing by the online career guidancesystem a plurality of normalized career skill levels for each career inthe plurality of careers with a corresponding plurality of normalizeduser skill levels in the plurality of normalized user skill levels;selecting by the online career guidance system a career in the pluralityof careers with a plurality of normalized career skill levels that mostclosely matches a corresponding plurality of normalized user skilllevels; displaying by the online career guidance system on the client tothe user a first polygon having a first plurality of vertices and afirst focus, wherein each vertex of the first polygon is spaced adistance from the first focus based on a normalized career skill levelin the plurality of normalized career skill levels for the selectedcareer; displaying by the online career guidance system on the client tothe user a second polygon, superimposed with the first polygon, having asecond plurality of vertices and a second focus, wherein the firstpolygon and the second polygon have the same number of vertices and thesecond focus is located at the same position on the client as the firstfocus, wherein each vertex of the second polygon is spaced a distancefrom the second focus based on a normalized user skill level in theplurality of normalized user skill levels for the selected career,wherein each vertex of the second polygon is located on a hypotheticalline defined by the first focus and a corresponding vertex of the firstpolygon and wherein each vertex of the first polygon or the secondpolygon is labeled with a career skill that corresponds with thenormalized career skill level and the normalized user skill level forthat vertex; determining by the online career guidance system a careerskill deficit between a normalized career skill level in the pluralityof normalized career skill levels for the selected career and acorresponding normalized user career skill level in the plurality ofnormalized user career skill levels; selecting an action by the onlinecareer guidance system that has previously been determined to reduce thecareer skill deficit; and displaying by the online career guidancesystem an action on the client to the user near the first polygon andthe second polygon.
 9. The method of claim 8, further comprising thesteps of: receiving by the online career guidance system a selection ofthe action on the client by the user; and performing by the onlinecareer guidance system a registration process of the user for theaction.
 10. The method of claim 8, wherein the action is an onlineeducational class.
 11. The method of claim 8, wherein the first polygonand the second polygon are irregular polygons.
 12. The method of claim8, wherein each career in the plurality of careers is associated withthree to eight different skills predetermined to be desired in thecareer.
 13. The method of claim 8, further comprising the steps of:determining by the online career guidance system a plurality ofnormalized prior user skill levels, wherein each normalized prior skilllevel is a skill level of the user at a same prior date; and displayingby the online career guidance system on the client to the user a thirdpolygon, superimposed with the first polygon and the second polygon,having a third plurality of vertices and a third focus, wherein thefirst polygon, the second polygon and the third polygon have the samenumber of vertices and the third focus is located in the same positionon the client as the first focus and the second focus, wherein eachvertex of the third polygon is spaced a distance from the third focusbased on a normalized prior user skill level in the plurality ofnormalized prior user skill levels for the selected career, wherein eachvertex of the third polygon is located on a hypothetical line defined bythe first focus and a corresponding vertex of the first polygon.
 14. Themethod of claim 8, further comprising the step of: displaying by theonline career guidance system a first plurality of polygons comprisingvertices that represent a normalized career skill level, wherein each ofthe first plurality of polygons represents a different career and issuperimposed with one of a corresponding second plurality of polygonscomprising vertices that represent a normalized user skill level,wherein each polygon, as compared to the other polygons, in the firstplurality of polygons comprises a plurality of vertices representing adifferent combination of career skills predetermined to be desired inthe career.
 15. A method of providing career guidance to a user,comprising the steps of: authenticating and logging by an onlineguidance system a user in to a user account; reading from a firstdatabase by the online career guidance system a plurality of userscores, wherein each user score is determined from a differentachievement of the user; converting each user score in the plurality ofuser scores into one or more user skill levels thereby generating aplurality of user skill levels, wherein each user skill level representsa predicted level of a skill of the user for a particular skillpredetermined to be desired in one or more careers; normalizing by theonline career guidance system the plurality of user skill levels to adesired scale to generate a plurality of normalized user skill levels;reading from a second database by the online career guidance system aplurality of careers and a plurality of career skill levels for each ofthe plurality of careers, wherein each career skill level for each ofthe plurality of careers represents a level of a skill predetermined tobe desired in the career; normalizing by the online career guidancesystem the plurality of career skill levels to generate a plurality ofnormalized career skill levels; displaying by the online career guidancesystem a plurality of criteria for selecting a career in the pluralityof careers; receiving by the online career guidance system a criteriaselected by the user from the plurality of criteria; comparing by theonline career guidance system a plurality of normalized career skilllevels for each career in the plurality of careers with a correspondingplurality of normalized user skill levels in the plurality of normalizeduser skill levels; selecting by the online career guidance system acareer from the plurality of careers based on the criteria selected bythe user and the comparing of the plurality of normalized career skilllevels with the corresponding plurality of normalized user skill levels;displaying by the online career guidance system on the client to theuser a first polygon having a first plurality of vertices and a firstfocus, wherein each vertex of the first polygon is spaced a distancefrom the first focus based on a normalized career skill level in theplurality of normalized career skill levels for the selected career;displaying by the online career guidance system on the client to theuser a second polygon, superimposed with the first polygon, having asecond plurality of vertices and a second focus, wherein the firstpolygon and the second polygon have the same number of vertices and thesecond focus is located at the same position on the client as the firstfocus, wherein each vertex of the second polygon is spaced a distancefrom the second focus based on a normalized user skill level in theplurality of normalized user skill levels for the selected career,wherein each vertex of the second polygon is located on a hypotheticalline defined by the first focus and a corresponding vertex of the firstpolygon and wherein each vertex of the first polygon or the secondpolygon is labeled with a career skill that corresponds with thenormalized career skill level and the normalized user skill level forthat vertex; determining by the online career guidance system a careerskill deficit between a normalized career skill level in the pluralityof normalized career skill levels for the selected career and acorresponding normalized user career skill level in the plurality ofnormalized user career skill levels; selecting an action by the onlinecareer guidance system that has previously been determined to reduce thecareer skill deficit; and displaying by the online career guidancesystem an action on the client to the user near the first polygon andthe second polygon.
 16. The method of claim 15, further comprising thesteps of: receiving by the online career guidance system a selection ofthe action on the client by the user; and performing by the onlinecareer guidance system a registration process of the user for theaction.
 17. The method of claim 15, wherein the action is an onlineeducational class.
 18. The method of claim 15, wherein the first polygonand the second polygon are irregular polygons.
 19. The method of claim15, wherein each career in the plurality of careers is associated withthree to eight different skills predetermined to be desired in thecareer.